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Neural networks are algorithms that attempt to mimic the brain by learning the 
behavior of data. They can be thought of as a multivariate non-linear fit of inputs 
to outputs. 

In this study a new extended Kalman filter (EKF) learning algorithm for feed- 
forward neural networks (FFN) is used. With the EKF approach, the training of 
the FFN can be seen as state estimation for a non-linear stationary process. The 
EKF method gives excellent convergence performances provided that there is 
enough computer core memory and that the machine precision is high. Neural 
networks are ideally suited to describe the spatial and temporal dependence of 
tracer-tracer correlations. The neural network performs well even in regions 
where the correlations are less compact and normally a family of correlation 
curves would be required. For example, the CH4-N20 correlation can be well 
described using a neural network trained with the latitude, pressure, time of year, 
and CH4 volume mixing ratio (v.m.r.). The neural network was able to reproduce 
the CH4-N20 correlation with a correlation coefficient between simulated and 
training values of 0.9997. 
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Abstract ACpD 

In this study a new extended Kalman filter (EKF) learning algorithm for feed-forward 4, 3653-3667, 2004 

neural networks (FFN) is used. With the EKF approach, the training of the FFN can be 





et al., 1 997; Morgenstern et al., 2002) and to distinguish between air-parcels of different 

origins (Waugh and Funatsu, 2003). The description of such spatially and temporally © EGU 2004 

3654 


dependent correlations are usually achieved by a family of correlations. However, a ACPD 

single neural network is a natural and effective alternative as shown by our previous 

study (Lary et al., 2004). 4 > 3653-3667, 2004 

This study uses the same dataset as Lary et al. (2004) but uses a quicker and more 
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prohibitive. Furthermore, it was noticed (Bierman, 1977) that round off errors due to 

poor computer precision can sometimes make the algorithm numerically unstable. © EGU 2004 
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The storage issue has been addressed by a number of research groups (Shah et a!., ACPD 

1992 ; Puskorius and Feldkamp, 1994 ). They attacked the estimation problem (train- 
ing) by partitioning it into a set of subproblems assuming the existence of mutually 4 - 3653 - 3667, 2004 

independent groups of weights; the numerical stability issue can be overcome by using 
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2.2. Training procedure using EKF ACPD 

The training (state estimation) is now a problem of determining w (the state vector) that 4, 3653-3667, 2004 

minimizes the sum of squared prediction errors of all observed data so far. 
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- The FNN had only one output, i.e. d y , R y , the innovation and the denominator in ACPD 

Eq. (3) are all scalars. 

4, 3653-3667, 2004 

- y j = h(w,x) = tanh[w(m(n + 1) + 1 : m + 1) z(1 : m + 1)], where z(2 : m + 1) = 





N 2 0 correlation with a correlation coefficient between simulated and training values of 

0.9995. So the new algorithm gives better results with much less expense. Figure 1b © EOU 2004 

3660 


shows a scatter diagram of the known N 2 0 concentration against the neural network ACPD 

N 2 0 concentration. Figure 1 c shows the way the rms error changes with epoch. 

Both CH 4 and pressure are strongly correlated with N 2 0 as can be seen in Fig. 1 of 4 > 3653 - 3667 , 2004 

Lary et al. ( 2004 ). Latitude and time are only weakly correlated with N 2 0 but still play 
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4 . Conclusions ACPD 

Neural networks are ideally suited to describe the spatial and temporal dependence of 4 , 3653-3667, 2004 

tracer-tracer correlations. Even in regions when the correlations are less compact. Us- 
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